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Cost  Growth,  Acquisition  Policy,  and  Budget  Climate* 

Contractors  involved  in  a  weapon  system  acquisition,  of  course,  perfonn  the  development 
and  manufacturing  work.  The  Department  of  Defense  (DoD)  Program  Manager  (PM)  and 
personnel  in  the  program  office  oversee  the  contractors  and  day-by-day  do  the  myriad  things  that 
must  be  done  for  a  major  acquisition  program  to  move  forward — funding,  contracting,  and 
planning,  among  many  others.  Clearly,  a  good  outcome  for  a  program  will  not  occur  if  these 
individuals  do  not  do  their  jobs  well.  It  is  equally  obviously  that  if  the  contractors  and  the 
government  personnel  do  their  jobs  well,  a  good  outcome  for  the  program  is  more  likely. 

What  this  focus  on  the  DoD  PM,  the  program  office  personnel,  and  the  contractors’  PMs 
and  workers  leaves  out  are  factors  they  must  accept  as  “givens.”  These  givens  are  subject  to 
changes — sometimes  large  and  fairly  sudden — that  presumably  have  substantial  consequences 
for  program  outcomes.  One  of  the  givens  is  DoD  acquisition  policy  and  process.  A  second  is  the 
topline  DoD  budget  constraint,  which  does  not  determine,  but  generally  has  a  marked  influence 
on,  the  funding  for  individual  programs. 

This  paper  summarizes  the  results  of  research  done  at  the  Institute  for  Defense  Analyses1  on 
whether,  taking  account  of  budget  climate,  changes  in  DoD  acquisition  policy  and  process  had  a 
discernible  effect  on  growth  of  Program  Acquisition  Unit  Cost  (PAUC)  of  major  defense 
acquisition  programs  (MDAPs).  The  paper  attempts  to  shed  some  light  on — and  as  it  turns  out,  to 
challenge — three  assumptions  that  often  are  implicit  in  discussions  of  acquisition  refonn: 

•  Cost  growth  and  other  problems  of  program  outcome  primarily  reflect  what  does  and 
perhaps  does  not  take  place  in  the  acquisition  realm. 

•  Changes  in  acquisition  policy  and  process  can  be  expected  to  have  substantial  effects  on 
the  cost,  schedule,  and  perfonnance  results  of  MDAPs. 

•  The  ingrained  cultures  of  DoD  acquisition  organizations  are  a  substantial  obstacle  to 
steps  that  might  reduce  cost  growth. 

While  not  organized  around  these  topics,  the  paper  provides  results  that  bear  directly  on  them. 

Acquisition  Regime  and  PAUC  Growth 

DoD  acquisition  policy  and  process  over  the  period  1970-2007  can  be  grouped  into  five 
successive  regimes: 

1.  The  Defense  Systems  Acquisition  Review  Council  (DSARC),  1970-1982 

2.  The  Post-Carlucci  Initiatives  DSARC,  1983-1989 

3.  The  Defense  Acquisition  Board  (DAB),  1990-1993 

4.  Acquisition  Reform  (AR),  1994-2000 

5.  The  DAB  -  Post  Acquisition  Reform,  2001-2007 


*  David  L.  McNicol,  Research  Staff  Member,  Institute  for  Defense  Analyses.  Valuable  comments  reflected  in  the 
paper  were  provided  by  Dr.  David  Sparrow,  Dr.  Brian  Rieksts,  and  Dr.  Prashant  Patel.  Dr.  Sarah  Burns  provided 
advice  and  assistance  on  the  statistical  analysis.  Ms.  Linda  Wu  managed  data  acquisition  and  the  database. 

1  David  L.  McNicol  and  Linda  Wu,  “Evidence  on  the  Effect  of  DoD  Acquisition  Policy  and  Process  on  Cost 
Growth  of  Major  Defense  Acquisition  Programs,”  IDA  Paper  P-5126.  Alexandria,  VA:  Institute  for  Defense 
Analyses,  2014  (forthcoming).  This  paper  was  sponsored  by  the  Director,  Performance  Assessments  and  Root 
Cause  Analyses  of  the  Office  of  the  Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics.) 


1 


Table  1  displays  the  average  PAUC  growth  for  MDAPs  that  passed  Milestone  (MS)  II/B  or 
filed  a  first  Selected  Acquisition  Report  (SAR)  in  each  of  these  regimes.  The  PAUC  growth 
figures  all  are  measured  from  the  MS  II/B  baseline  and  nonnalized  to  the  MS  II/B  total  inventory 
objective.  There  are  a  number  of  interesting  aspects  to  these  data;  for  example,  the  high  PAUC 
growth  during  the  AR  period  and  the  lower  PAUC  growth  for  2001-2007.  Granting  that,  the 
single  most  notable  feature  of  these  data  is  the  absence  of  any  trend  in  PAUC  growth.  If  changes 
in  acquisition  policy  and  process  have  had  a  sustained  influence  on  PAUC  growth,  it  does  not 
show  up  in  these  data. 

Broadly,  there  are  two  ways  to  explain  the  absence  of  sustained  effects  of  acquisition  policy 
and  process  on  the  PAUC  growth  data.  First,  they  may  in  fact  not  have  a  strong  or  consistent 
effect  on  PAUC  growth.  Second,  acquisition  policy  and  process  may  have  substantial  effects  that 
are  masked  by  some  other  factor  or  factors. 


Table  1.  Average  PAUC  Growth  in  Successive  Acquisition  Regimes 


Acquisition  Regime 

Time  Period 

Average  PAUC 
Growth 

No.  of 

Observations 

DSARC 

1970-1982 

32% 

48 

Post  Carlucci  Initiatives  DSARC 

1983-1989 

19% 

40 

DAB 

1990-1993 

36% 

11 

Acquisition  Reform  (AR) 

1994-2000 

61% 

27 

DAB  post  AR 

2001-2007 

19% 

25 

Budget  Climate  and  PAUC  Growth 

Thinking  along  the  lines  of  the  second  of  these  possibilities  led  to  consideration  of  whether 
changes  in  the  DoD  budget  climate  might  be  associated  with  PAUC  growth.  The  period  1970- 
2007  includes  two  sub-periods  during  which  the  DoD  topline  was  contracting  or  generally 
considered  to  be  “tight”:  1970-1978  and  1987-2001.  It  also  includes  two  sub-periods  in  which 
the  DoD  topline  was  growing  or  thought  to  be  “high”:  1979-1986  and  2002-2007.  Table  2 
displays  the  average  PAUC  growth  data  for  these  four  sub-periods. 


Table  2.  Average  PAUC  Growth  in  Different  DoD  Topline  Conditions 


DoD  Topline  Contracting  or  Tight 

DoD  Budget  Growing  or  High 

Period 

PAUC  Growth 

Period 

PAUC  Growth 

1970-1978 

37%  (38) 

1979-1986 

12%  (39) 

1987-2001 

51%  (53) 

2002-2007 

10%  (21) 

Note:  Numbers  in  parentheses  are  the  number  of  observations  available. 


These  data  make  it  clear  that  the  average  PAUC  growth  in  Contracting/Tight  topline 
periods  is  far  larger  than  it  is  in  periods  during  which  the  topline  was  Growing/High — by  a  factor 
of  three  in  the  first  comparison  and  by  more  than  a  factor  of  five  in  the  second.  This  is  clearly  an 
effect  potentially  strong  enough  to  mask  other  influences. 
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Acquisition  Regime  and  Budget  Climate 

Table  3  expands  Table  2  by  replacing  the  budget  climate  sub-periods  with  the  acquisition 
policy  and  process  regimes.  This  table  provides  results  for  two  sets  of  natural  experiments.  First, 
the  PAUC  growth  columns  give  the  effect  of  changes  in  the  acquisition  regime  for  a  given 
budget  climate.  Second,  the  rows  show  the  effect  of  budget  climate  for  a  given  acquisition 
regime.  For  example,  the  first  nine  years  of  the  DSARC  (1970-1978)  were  in  a  tight  budget 
climate,  while  the  next  four  (1979-1982)  were  in  a  period  in  which  the  DoD  topline  was 
growing. 


Table  3.  Average  PAUC  Growth  by  Acquisition  Regime  and  Topline  Condition 


Acquisition  Regime 

Topline  Contracting/Tight 

Topline  Growing/High 

Period 

PAUC  Growth 

Period 

PAUC  Growth 

DSARC 

1970-1978 

37%  (38) 

1979-1982 

10%  (10) 

Post  Carlucci  DSARC 

1987-1989 

34%  (11) 

1983-1986 

13%  (29) 

DAB 

1990-1993 

36%  (11) 

None 

N/A 

Acquisition  Reform  (AR) 

1994-2000 

61%  (27) 

None 

N/A 

DAB  post  AR 

2001 

66%  (4) 

2002-2007 

10%  (21) 

Note:  Numbers  in  parentheses  are  the  number  of  observations  available. 


Statistical  analysis  of  the  averages  in  Table  3  leads  to  two  conclusions.  First,  there  is  no 
statistically  significant  improvement  or  worsening  of  PAUC  growth  correlated  with  the  different 
acquisition  policy  regimes.  This  is  obvious  for  the  Growing/High  climate  (column  on  the  right  in 
Table  3).  In  contrast,  PAUC  growth  over  1994-2000  and  in  2001  (column  on  the  left  in  Table  3) 
is  noticeably  higher  than  the  averages  for  previous  periods,  but  the  differences  proved  not  to  be 
statistically  significant. 

Second,  PAUC  growth  tends  to  be  substantially  higher  in  a  Contracting/Tight  budget 
climate  than  in  the  Growing/High  climate.  We  have  only  three  natural  experiments  of  changes  in 
budget  climate  for  a  given  acquisition  regime,  since  two  of  the  five  acquisition  regimes  (DAB 
and  AR)  fall  entirely  within  one  budget  climate — Contracting/Tight.  Each  of  these  three  natural 
experiments  on  the  effect  of  budget  climate  has  the  same  outcome — moving  from  a 
Contracting/Tight  budget  climate  to  a  Growing/High  budget  climate  for  a  given  acquisition 
regime  is  associated  with  much  lower  cost  growth.  The  outcomes  of  the  first  two  experiments  are 
virtually  identical — an  average  PAUC  growth  of  37  and  34  percent,  respectively,  in  the  two 
periods  when  the  topline  was  Contracting/Tight  and  average  PAUC  growth  of  10  percent  and  13 
percent,  respectively,  in  the  two  periods  when  the  topline  was  Growing/  High.  The  effect  is  most 
pronounced  in  the  third  experiment  (DAB  post-AR),  which,  with  the  first,  is  statistically 
significant. 

There  is  a  further  important  point  in  the  data  behind  Table  3.  PAUC  growth  is  not  a 
problem  of  the  typical  MDAP  in  periods  of  Contracting/Tight  budget  climate,  but  of  the  minority 
of  MDAPs  that  experienced  high  cost  growth.  This  conclusion  rests  primarily  on  three  points: 

•  In  periods  of  Contracting/Tight  budget  climate,  about  40  percent  of  MDAPs  had  a 
PAUC  growth  of  at  least  50  percent. 
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•  These  MDAPS  had  an  average  PAUC  growth  of  92  percent,  accounting  for  78  percent 
of  total  PAUC  growth. 

•  Average  PAUC  growth  in  these  periods,  excluding  high  cost  growth  MDAPs  and 
MDAPS  with  negative  PAUC  growth,  was  2 1  percent. 

This  conclusion  is  important  because  it  suggests  the  possibility  that  reforms  directed  to  the 
average  or  typical  MDAP  may  miss  the  real  source  of  the  problem. 

Does  the  Resource  Allocation  Process  Play  a  Major  Role  in  PAUC  Growth? 

These  conclusions  tend  to  challenge  a  fundamental  assumption  implicit  in  most  discussions 
of  acquisition  reform:  that  the  main,  although  not  the  only,  causes  of  PAUC  growth  are  to  be 
found  in  the  acquisition  realm — the  effectiveness  of  the  PM,  the  adequacy  of  the  developmental 
test  plan,  the  reasonableness  of  the  cost  estimate,  the  completeness  of  the  systems  engineering 
plan,  among  many  others.  This  assumption  is  hard  to  maintain  when  the  many  changes  in 
acquisition  policy  and  process  made  in  the  past  four  decades  have  not  had  statistically  significant 
effects  on  PAUC  growth,  but  there  is  a  significant  association  between  PAUC  growth  and  budget 
climate  at  the  point  when  the  original  baseline  was  set. 

The  association  between  PAUC  growth  and  budget  climate  suggests  that  the  resource 
allocation  process,  particularly  at  the  Service  level,  plays  an  important  role  in  cost  growth.  This 
does  not  mean  “budget  instability.”  Budget  instability  is  a  term  of  art  for  changes  in  MDAP 
funding  through  the  annual  resourcing  cycle  and  “taxes.”  Budget  instability  is  a  chronic 
condition,  present  to  some  degree  in  all  periods,  and  affects  program  execution  but  not  problems 
of  program  inception.  What  this  paper  observed  is  a  recurring  pattern — that  MDAPs  that  passed 
MS  II/B  during  periods  when  the  DoD  topline  was  contracting  or  tight,  on  average,  had  much 
higher  PAUC  growth  than  those  that  passed  MS  II/B  when  the  topline  was  growing  or  high. 

The  conjecture  that  the  resource  allocation  process  plays  an  important  role  in  cost  growth 
gets  some  support  from  an  unexpected  direction — MDAPs  with  negative  cost  growth,  of  which 
there  are  twenty-nine  in  our  sample.  Negative  PAUC  growth  is  recorded  if  the  actual  cost  of  a 
program  proves  to  be  less  than  the  cost  in  the  MS  II/B  baseline.  Assuming  the  program  was 
funded  to  its  MS  II/B  baseline,  this  implies  that  over  time  funds  can  be  taken  from  the  program 
in  question  and  reallocated  to  other  applications,  including  other  acquisition  programs.  The 
program,  then,  effectively  can  be  used  as  a  “bank” — a  way  to  hold  reserves  in  relative  safety 
until  they  are  needed.  A  bank  of  this  sort  is  more  likely  to  be  needed  in  a  Growing/High  budget 
climate,  as  it  can  then  serve  as  a  way  to  delay  final  decisions  on  the  higher  level  of  funding  that 
has  become  available.  We  would  therefore  expect  to  find  relatively  more  instances  of  negative 
PAUC  growth  in  the  Growing/High  budget  periods,  and  this  is  what  we  observe.  About  30 
percent  of  our  observations  in  Growing/High  budget  climates  are  of  negative  PAUCs,  compared 
to  about  10  percent  across  the  periods  of  Contracting/Tight  climate. 

MDAPs  with  “high  cost  growth,”  which  we  define  as  quantity  normalized  PAUC  growth  of 
at  least  50  percent,  also  suggest  an  influence  from  resource  considerations.  DoD  resource 
managers,  particularly  at  the  Service  level,  have  only  a  few  tools  for  responding  to  a  Contracting/ 
Tight  budget  climate.  One  of  these  is  to  impose  top-down  limits  on  the  funding  for  particular 
MDAPs  as  they  approach  MS  II/B.  Plausibly,  the  result  will  be  particularly  optimistic 
programmatic  and  costing  assumptions,  which  lead  to  an  expectation  that  relatively  more  high 
cost  growth  programs  will  be  started  in  periods  of  Contracting/Tight  budget  climate.  This  is  what 
is  observed.  During  periods  of  Contracting/Tight  budget  climate,  about  40  percent  of  MDAPs 
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had  very  high  PAUC  growth.  In  contrast,  during  periods  of  Growing/High  budget  climate  only 
about  7  percent  of  MDAPs  experienced  high  PAUC  growth. 

Implications  for  Discussions  of  Acquisition  Reform 

This  paper  points  to  three  implications  for  a  discussion  of  acquisition  refonn.  First,  the 
relevant  context  for  understanding  PAUC  growth  is  the  interface  between  the  acquisition  process 
and  the  resource  allocation  process.  The  crucial  evidence  behind  this  point  is  the  strong 
association  between  budget  climate  and  PAUC  growth.  Resource  managers  must  think  in  terms 
of  a  portfolio  of  programs  across  mission  areas  and  commodity  types,  and  extending  from  efforts 
in  the  technology  base  through  programs  nearing  the  end  of  production.  When  a  program  is 
completed,  it  opens  a  resource  “hole”  that  programs  emerging  from  Engineering  and 
Manufacturing  Development  can  occupy.  In  turn,  programs  earlier  in  the  acquisition  cycle  can 
move  forward  as  well.  When  funding  for  acquisition  turns  down,  these  holes  get  smaller,  or  close 
entirely,  or  require  cuts  in  funding  for  ongoing  programs.  The  alternatives  available  in  this 
circumstance  are  all  undesirable — cancellations  of  programs,  delays  in  new  starts,  stretches,  and 
unrealistic  pricing.  The  evidence  summarized  here  suggests  that  it  is  in  this  context  that  high 
PAUC  growth  arises. 

Second,  it  seems  unlikely  that  further  broad  changes  in  the  acquisition  process  would  have 
a  major  effect  on  PAUC  growth.  The  research  found  no  evidence  that  the  efforts  to  strengthen 
the  acquisition  process  through  the  years  have  resulted  in  lower  or  higher  PAUC  growth.  This 
does  not  mean  that  the  DAB  process  does  not  provide  a  useful  discipline  on  acquisition 
programs;  moreover,  further  changes  in  acquisition  policy  or  process  might  be  warranted  for 
reasons  of  good  government.  The  evidence  does,  at  a  minimum,  suggest  that  the  effects  of 
changes  in  the  acquisition  process  since  the  early  1970s  have  not  had  a  dominant  effect  on 
PAUC  growth. 

Third,  it  is  difficult  to  see  that  the  cultures  of  the  DoD  acquisition  organizations  are  a 
crucial  obstacle  to  improved  performance  on  cost  growth.  The  key  point  to  note  is  that  high 
PAUC  growth  is  not  persistent,  but  rather  episodic,  and  correlated  with  environmental  factors 
outside  of  the  control  of  the  acquisition  process.  There  is  remarkably  little  PAUC  growth  in 
periods  when  the  budget  is  Growing/High.  It  seems  fair  to  ask  if  it  makes  sense  to  assert  that  an 
entrenched  culture  sometimes  results  in  high  cost  growth  and  other  times  in  low  cost  growth.  Just 
how  is  it  that  the  A  team  takes  the  field  so  quickly  and  quietly  when  the  budgetary  sun  comes 
out?  And  why  even  in  bad  budgetary  weather  do  more  than  half  of  MDAPs  exhibit 
comparatively  modest  PAUC  growth? 
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(MDAPs).  Examination  of  PAUC  growth  data  for  150  MDAPs  that  passed  Milestone  II  or  B  during  1970-2007  does  not  reveal  any 
substantial  or  consistent  effect  of  changes  in  acquisition  policy  and  process.  Changes  in  budget  climate,  however,  are  found  to  have 
a  large  influence  on  PAUC  growth.  These  findings  have  three  implications  for  acquisition  reform.  First,  the  underlying  cause  of 
persistent  high  PAUC  growth  is  not,  as  is  commonly  asserted,  a  deeply  established  culture  of  the  DoD  acquisition  organizations  and 
their  professional  employees.  Second,  it  seems  unlikely  that  further  changes  in  the  acquisition  process  will  have  a  major  effect  on 
PAUC  growth.  Third,  the  relevant  context  for  understanding  PAUC  growth  is  the  interface  between  the  acquisition  process  and  the 
program/budget  process. 
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